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ABSTRACT 



First Steps is a program instituted by the Western Australia 
Ministry of Education to improve the literacy and numeracy of primary school 
students, particularly those at risk of academic failure. First Steps is 
organized around reading, writing, spelling, and oral language; children's 
skills are organized along developmental continua. These continua are an 
ordered series of statements describing behaviors children exhibit as they 
develop literacy skills. This report studies the Reading Developmental 
Continuum empirically and presents the responses of classroom teachers toward 
the continuum. Data for the validation of the continuum were provided by 
teachers of kindergarten, and Years 1, 3, 5, and 7 from a sample of Western 
Australia schools. Useable data were provided by 36 teachers, each of whom 
evaluated the position of up to 10 students in their class according to the 
continuum. Teachers also responded to questions about the indicators for the 
continuum. Overall, there was a high level of understanding of the 
indicators. The Reading Development Continuum was thought to depict the order 
of development of literacy accurately, with two problem areas identified: (1) 

there was overlap in the estimated difficulty of indicators in the Role Play 
and Experimental phases; and (2) the key indicators of the Role Play Phase 
were deemed more difficult than those for the development ally later 
Experimental Phase. Teachers also reported that the First Steps continua were 
either helpful or very helpful to them. Four appendixes contain additional 
data about teacher responses to the indicators of the continuum and the 
teacher questionnaire. (Contains 12 exhibits and 6 references.) (SLD) 
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SUMMARY OF THE MAIN FINDINGS 



1. There is a high level of understanding of the indicators from the First Steps Reading 
Developmental Continuum by the Kinder, Year 1, 3, 5 and 7 classroom teachers 
participating in this evaluation. 

2. The Reading Development Continuum, in general, validly depicts the order of the 
development of children's reading abilities. However, two problems are identified. 
These are: 

• there is considerable overlap in the estimated difficulty of the indicators in 
the Role Play Phase and the indicators in the Experimental Phase and, 

• the key indicators in the Role Play Phase are, on average, more difficult than 
the key indicators in the developmentally later Experimental phase. 

3. Most key indicators in the Reading Development Continuum are shown to have 
been appropriately defined by First Steps as 'key indicators'. A small number, 
particularly one in the Role Play Phase, do not seem to be tapping reading ability as 
well as other indicators. This is a concern. 

4. Generally, where problems are identified with the Reading Developmental 
Continuum, it is in the Role Play Phase. 

5. Classroom teachers, forming the sample used for this report, typically regard the 
Reading or Writing or Spelling or Oral Language First Steps Developmental Continua 
as accurately depicting the development of literacy in the children that they teach. 
(Note that the sample was not drawn randomly, so this finding is not generalisable. 
The finding is, however, consistent with other research findings.) 

6. Nearly all teachers in the sample reported that the First Steps Continua are either 
helpful or very helpful to them. (Again, note that this is not a generalisable finding, 
although, it is consistent with other research findings.) 
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INTRODUCTION 1 



Background to the First Steps Project 

First Steps is a program instituted by the Western Australian Ministry of Education to 
improve the literacy and numeracy of primary school students. It is intended, in 
particular, to assist in the development of the literacy and numeracy skills of 'at risk' 
students. First Steps was introduced in 1989 and has been evolving since that time. 
Most of the work so far produced by First Steps, has focused upon literacy. 

First Steps sees the acquisition of literacy as an integrated process. It claims, for 
example, that; 



"Language development cannot be divided into 
discrete components. Reading, writing, speaking 
and listening are interrelated. 

* They parallel each other 

* They complement each other. 

* They support each other. 

* They very often occur together." 

(Writing Development Continuum, p. v) 

Nevertheless, First Steps holds the view that for teaching purposes it is often necessary 
to focus on particular aspects of language and literacy. This seems a reasonable view. 
The process of becoming literate, indeed of teaching language skills, is complex and if 
teaching strategies and methods are to be applied then they will require some form of 
organisation. First Steps organises aspects of literacy around four themes. 

The four themes around which First Steps organises its literacy program are 'Reading', 
'Writing', 'Spelling' and 'Oral language'. 

Each of the four themes is, in turn, organised around a 'developmental continuum'. 
These Continua consist of an ordered series of statements describing behaviours 
children exhibit as they develop literacy skills. These statements are regarded by First 



^ If you have read the earlier report Empirical Validation of the First Steps Spelling and Writing 
Continua then you will quickly see that the introductory notes used in this report are very 
closely based upon this earlier report. If you have read the earlier report you may wish to skip 
to where the discussion of the results from the data analysis begins. 



Steps as akin to milestones marking out a child's development along the road to 
literacy . 2 

During the construction of these Continua, it was observed by First Steps staff that 
various indicators clustered together. These clusters of indicators were incorporated 
into the structure of the Continua and were named 'phases'. For example, in the 
Reading Developmental Continuum there were six phases identified. These were 
named 'Role Play Reading Phase', 'Experimental Reading Phase', 'Early Reading 
Phase', Transitional Reading Phase', 'Independent Reading Phase' and 'Advanced 
Reading Phase'. There are differing numbers of indicators in each phase. 

Thus, each Continuum consists of a small number of phases. Within each phase are 
statements (named 'indicators') that describe various behaviours associated with 
acquiring literacy. As well, all phases have some indicators that have been judged to 
be more important than other indicators within that phase. These more important 
indicators are named 'key indicators'. Key indicators were identified by First Steps 
personnel during research conducted as part of the development of the First Steps 
program. 

First Steps proposes that the Continua allow a teacher to 'locate' where a child is 'at' in 
his or her development of literacy skills. Once this location is identified, the most 
appropriate strategies for that phase of development can be used in the classroom. 
(First Steps provides many strategies, each linked to the various phases of 
development.) 

The location of a child on a Continuum is established by using the key indicators. If a 
child has exhibited all the key indicators within a phase, then that child is said to be 
within that phase. In this way, the phases are used to locate a child on a Continuum. 
When a child exhibits all the key indicators in the next phase of development, he or 
she is then said to have moved into the next phase of development. For example, a 
child who has exhibited all the key indicators in the Experimental Reading Phase will 
remain in that phase until he or she has exhibited all the key indicators in the Early 
Reading Phase. 




More precisely, the First Steps Developmental Continua represent the milestones of children's 
development in English speaking countries where a Western school system operates. This 
includes such countries as Australia, Canada, New Zealand, United Kingdom and the United 
States. The research drawn on during the construction of the First Steps continua was largely 
from these countries. Until there is evidence which shows the continua generalise beyond these 
school systems, it is probably better to approach their use in other school systems with caution. 
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First Steps suggests that, at any given time, a child will probably exhibit behaviours 
across several of the phases. This occurs because of variation in children's 
development. However, it is claimed that, overall, the Continua do depict the typical 
patterns of development to be found in children. 



Background to the evaluation of First Steps 

The WA Ministry of Education approached ACER in early 1992 to evaluate First Steps. 
Work for the evaluation began in April 1992. Three reports have so far been produced 
as part of the evaluation. These are; The Impact of First Steps on Schools and Teachers 
and The Impact of First Steps on the Reading and Writing ability of Year 5 students, and the 
Empirical Validation of the First Steps Spelling and Writing Continua. 

This report describes how the Reading Continuum was examined (or 'validated'). 

Aims of this report 

This report has two aims. The first is to empirically validate the Reading 
Developmental Continuum. The second is to report how classroom teachers view the 
validity of the First Steps Continua. 



Structure of the Report 

The report has two main parts. The first deals with the empirical validation of the 
Reading Continuum. The second part deals with the responses of classroom teachers 
to this Continuum. This second part is based upon responses taken from 
questionnaires sent to teachers as part of the evaluation. 
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Editor’s Note 



This document is one of a series of reports that document the formative research that supported 
the creation and development of First Steps™. As a result of this research, the Education 
Department of Western Australia (EDWA), in collaboration with the Australian Council for 
Educational Research (ACER) revised First Steps in response to each of the issues and questions 
raised by this research. First Steps training courses, Developmental Continua, and Resource 
Books are published with due amendments and alterations. 

Other research documents that support the development of First Steps include: 

Dr. Phil Deschamp: 

♦ A Survey of the Implementation of the Literacy Component of the First Steps Project in WA 

♦ The Implementation of The Literacy Component of The First Steps Project in ELAN Schools 

♦ A Survey of the Effectiveness of the Focus Teacher ‘B’ Training for the First Steps Project 

♦ Student Achievement: A Study of the Effects of First Steps Teaching on Student 
Achievement 

♦ Case Studies of The Implementation of the First Steps Project in Twelve Schools 

♦ The Development and Implementation of the First Steps Project in Western Australia 



ACER: 

♦ Empirical Validation of the First Steps Reading Continuum 

♦ Empirical Validation of the First Steps Spelling and Writing Continua 

♦ Empirical Re-Validation of the First Steps Spelling Continuum 

♦ Assessment and Record of the Changes made to the Spelling Continuum 

♦ The Impact of First Steps on Schools and Teachers 

♦ The Impact of First Steps on the Reading and Writing Ability of Year 5 Students 

♦ Background: First Steps and the ACER Evaluation & Report on the Validity of the First 
Steps Writing and Spelling Continua* 



EDWA: 

♦ Supporting Linguistic and Cultural Diversity Through First Steps: The Highgate Project 



For more information about on-going First Steps research, please contact: 

First Steps™ / Heinemann 
361 Hanover Street 
Portsmouth, NH 03801-3912 
1.800.541.2086, ext. 281 
firststeps@heienmann.com 
www.heinemann.com/firststeps 




11 



PART 1: THE EMPIRICAL VALIDATION OF THE FIRST STEPS READING 
DEVELOPMENTAL CONTINUUM. 



The research objective 

The objective of the empirical validation of the Reading Development Continuum is to 
collect information about the typical sequence in which students leam and to compare 
this with the sequence of learning proposed in the Reading Development Continuum. 

The research is not concerned to establish the validity or the meaning of the concepts 
used to describe the indicators. The research reported here cannot answer questions 
about whether, say, an indicator from the Role Play Phase such as The child uses 
pictorial cues' is meaningful or accurate. (It may, however, alert users to problems 
which arise because of incoherence or inaccuracy in an indicator.) As the indicators are 
drawn from empirical research the question of their accuracy and meaning needs to be 
addressed using other methods. This research is designed to find out about the 
ordering of the indicators of the Reading Development Continuum as proposed by 
First Steps. 

The Reading Developmental Continuum will be regarded as validated if the order of 
the indicators proposed by First Steps matches the order of development that teachers 
see in their students. 

The research questions 

The following specific questions are addressed in this report: 

Which First Steps indicators do teachers not understand? 

Are the indicators within a phase at about the same level of difficulty? 

Do the phases reflect a sequence which implies increasing difficulty? 

How 'key' are the 'key indicators'? 

The Sample 

The data for the validation of the Reading Developmental Continuum were provided 
by teachers of Kindergarten, Years 1, 3, 5 and 7 students from a sample of Western 
Australian government primary and District High schools. 
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The sample was not randomly drawn. As the Reading Developmental Continuum is 
not yet widely used by schools, it was decided to approach those schools which were 
thought to be most involved with it. These schools were identified by staff in the 
regional offices of the Ministry. 

Some schools which have had limited involvement with the Reading Development 
Continuum were inadvertently approached to participate in the validation. Some of 
these, despite their relative unfamiliarity with the Continuum, agreed to participate in 
the research. Thus not all the data come from teachers who were widely experienced 
with this particular continuum. It is not clear what effect this has had on the quality of 
the data. It has been assumed that the effect is minimal. 

Altogether, 54 teachers agreed to participate in this part of the evaluation. Of these 54 
teachers, 36 returned data that could be used. Exhibit 1 shows the numbers of teachers 
who responded for each Year level. 

Exhibit 1: Number of teachers providing data for the validation of the Reading 
Developmental Continuum for each Year level. 

Kinder Yrl Yr3 Yr 5 Yr7 Total 

4 9 8 9 6 36 



The Data 

The data consist of judgements made by teachers about the extent to which each of up 
to ten students in their class exhibited evidence of having demonstrated the behaviour 
described by each of a number of First Steps indicators. (The teachers made these 
judgements about each child in turn and not about the group of children.) These 
judgements were recorded on a computer used by the teachers. They tapped one of a 
set of appropriate keys to register their response. There were two sets of responses 
available to a teacher depending upon which indicator was displayed to them. The 
response set to be used was pre-determined. Teachers could not choose which 
response set to use. 

The first response set was: 

Y - (Yes/ Most of the time) 

This key was to be pressed if the named student usually demonstrated this 
behaviour. 




13 



6 



N -(No/Hardly Ever) 

This key was to be pressed if the named student did not or hardly ever 
demonstrated this behaviour. 

U - This key was to be pressed if the teacher was unable to make a judgement. 
If they responded with TJ', the program asked the teacher to select one of the 
following: 

1*1 have not yet had an opportunity to gather information relating to this 
indicator. 

2-1 don't understand the wording of this indicator. 

If the teacher entered 'U' then the program displayed all students' names, and beside 
them the response - 'U'. That is, if the teacher did not understand the indicator for one 
student, it was assumed that they did not understand it for all students. If the teacher 
had understood the indicator, then after all students had been evaluated with respect 
to that indicator, the results of the teacher's judgements were displayed on the screen 
and an opportunity was given to alter the data. 

The use of the first response set was not appropriate for all indicators. If this response 
set, and only this response set, had been used, then it would have led to ambiguous 
responses for some indicators. For example, consider the indicator from the Early 
Reading Phase of the Reading Developmental Continuum: 'The child points as an aid 
to reading ... ' (code number 2324, Appendix A). A response 'Hardly ever' could mean 
that the child 'Hardly ever points' because he or she is yet to leam how to do this. But 
the response 'Hardly ever' could also mean that the child 'Hardly ever points because 
he or she has advanced beyond this level of reading.' To overcome this problem, a 
second response set was developed. This second response set has the additional 
category of 'Beyond'. 

The second response set was: 

B - (Beyond this level.) 

This key was to be pressed if (1) the indicator was phrased negatively and the 
student demonstrated evidence of being able to perform the converse of the 
indicator, (2) the indicator began with the phrase "Beginning to ..." and the 
student had fully acquired the skill referred to in to the indicator or (3) a 
student had developmentally 'left behind' the indicator and so no longer 
demonstrated evidence of it (as opposed to having yet to develop this skill). 

O 
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Y - (Yes/Most of the time) 

This key was to be pressed if the student usually demonstrated this behaviour. 
N -(No/Hardly Ever) 

This key was to be pressed if the student did not, or hardly ever, demonstrated 
this behaviour and was yet to move beyond the level required to demonstrate 
competence on this indicator. 

U - This key was to be pressed if the teacher was unable to make a judgement. 
(The same categories - 1 or 2 - as for the first response set were then displayed.) 

In the data analysis Tes' and 'Beyond' were treated as having identical meaning and 
so coded to the same value. (Responses with the value TJ1' and TJ2' - unable to make 
a judgement - were excluded from the analysis which estimated the difficulty of the 
indicators.) 

Teachers were asked to provide data for the first five female students on a class list 
and for the first five male students on a class list. If a class had less than five female 
students or less than five male students then teachers were asked to 'top up' with 
other students from the class. In some small schools and in some composite classes 
there were less than ten students at a given Year level. In these cases, teachers were 
asked not to top up with students from other Year levels. 



Exhibit 2: Number of students involved in the study of the validation of the Reading 
Developmental Continua for each Year level. 

Kinder Yrl Yr3 Yr5 Yr7 Total 



31 85 80 69 54 319 



The data for the validation of the Reading Developmental Continuum were collected 
in November 1993. 

In summary, the data consist of judgements made by classroom teachers about the 
literacy behaviours of selected students on a number of different First Steps indicators. 




15 



8 



The Design of the Research 



It was decided to design the data collection in such a way as to avoid asking teachers 
about indicators which described behaviours that would be unlikely to be observed in 
their students. For example. Year 1 teachers were not asked to provide data about the 
indicators in the Transitional or Independent Reading Phases because it was felt to be 
most unlikely that any Year 1 students would exhibit any of these reading behaviours. 
No teachers were asked to provide data about the Advanced Reading Phase because it 
was judged that no primary school child would be in this Phase. 

To ensure that the data could still be used to depict a sequence of development across 
all phases of a Continuum, each Year level had at least one phase in common with the 
Year level below it or with the Year level above it. Exhibit 3 shows which phases of 
the Reading Developmental Continuum were used for each Year level, how these 
phases overlapped and how many indicators are in each phase. For example, in 
Exhibit 3, the column under the title Tr V indicates that Year 1 teachers providing 
data on the Reading Developmental Continuum, had indicators drawn from the 'Role 
Play', the 'Experimental' and the 'Early Reading' Phases. This means that for these 
Year 1 teachers there were 74 (21+ 26+ 27) indicators judged per child. 



Exhibit 3: Number of indicators per phase of the First Steps Reading Developmental 
Continuum for each Year level and distribution of phases across Year levels. 



Reading Continuum 
Phases 


N. of 

Indicators 


Kinder 


Yrl 


Yr 3 


Yr 5 


Yr 7 


Advanced 


22 












Independent 


19 








V 


V 


Transitional 


23 






V 


V 


V 


Early Reading 


27 


V 


V 


V 






Experimental 


26 


V 


V 








Role Play 


21 


V 










Total N of 
Indicators 


138 


74 


53 


50 


42 


42 
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Teachers were allocated phases according to the Year level that they taught. Phases 
were matched to Year level on advice from the First Steps project personnel. The 
matching was designed to ensure that the chosen phases were appropriate to the level 
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of development of the students. Inappropriate phases were, either, ones that 
contained indicators which all students of a given Year level would exhibit, or that 
contained indicators which no students, of a given Year level, would exhibit. A mix of 
student abilities was needed. 

Teachers providing data about the reading Developmental Continuum made, 
depending on the Year level taught and the number of students assessed, between 
about 420 and 740 judgements upon the reading behaviours of their students. 



Method of Data Collection 

All teachers were sent a computer disk. On this disk was a computer program written 
by staff at ACER. When the program was run it prompted teachers for responses to 
questions. These responses were stored on the disk and when the teacher had entered 
the data, the disks were returned to ACER. Each teacher received a disk containing 
the teacheri s name. Year level, and the indicators for the phases of the Continuum for 
which they would be providing data. 

When the program was run the teacher was first asked to enter the names of the 
students to be used, that is, the names of the first five boys and the first five girls on a 
class list. (Where there were fewer than five boys or five girls alternatives, described 
above, were adopted.) Student names were needed so that the teacher could be 
prompted for each specific child. The names of the teachers and the children have 
been removed from the data at ACER and no record has been retained of them. 

Teachers were next asked the following questions about each child: 

What is the child's sex? (M/F) 

Is English the first language of the child? (Y/N) 

Is the child an Aboriginal or Torres Strait Islander? (Y/N) 

Is the child receiving English as a Second Language assistance? (Y/N) 

Does the child have a disability that could significantly affect achievement in 
English? (Y/N) 

Once this was done, the teacher was presented the text of an indicator with the 
instruction to assess each student with respect to the indicator on display. The 
indicators were presented to the teachers in random order. The teachers knew only 
that the indicators were from Reading Development Continuum and that the 
indicators came from one of two or three phases. They did not know, unless they 
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recalled it from their own use of the Continuum, from which phase a displayed 
indicator came. Nor did they know if an indicator was a key indicator. 

It was estimated that teachers would take up to two hours to make all their 
judgements and enter the data. The computer program was designed so that teachers 
could quit before completing all the data entry and resume later. 



Method of Data Analysis 

The data for the empirical validation were analysed using the computer program 
"Quest" (Adams and Khoo, 1992) which produces Item Response Theory calibrations 
of indicators and measures of student achievement. An outline of this approach is 
provided below. 



Data Analysis 

The first step was to analyse data that had been collected about the students in the 
study. If these children had characteristics which suggested that their development of 
literacy skills might occur in a different order from most other students, then it was 
important to know this because it could effect the interpretation of results. 

It was possible to remove non-typical children from the analysis in an attempt to avoid 
any possibly distorting effects on the results. However, it was decided to retain all 
children because First Steps was instituted to assist precisely those children whose 
non-typicality placed them 'at risk'. Also, keeping all these students in the analysis 
means that the results are based upon a group of students who might typically make 
up a classroom. 

Attributes of the Children 

1. Sex 

It is generally the case that girls are more precocious than boys in the acquisition of 
literacy skills. It was therefore felt important to describe the distribution of the sex of 
the students. Exhibit 4 shows this for each Year level for the Reading Developmental 
Continuum. 
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Exhibit 4: Frequency and percentage of students involved in the study of the Reading 
Developmental Continuum by sex for each Year level. 





Kinder 


Yr 1 


Yr 3 


Yr 5 


Yr 7 Total 


Female 

Male 


16 (52%) 
15 (48%) 


44 (52%) 
41 (48%) 


37 (46%) 
43 (54%) 


40 (58%) 
29 (42%) 


29(54%) 166(52%) 
25(46%) 153(48%) 


Total 


31 (100%) 


85 (100%) 


80 (100%) 


69(100%) 


54 (100%) 319(100%) 



Exhibits 4 shows that the number of boys and girls is similar for each Year level for the 
Reading Development Continuum. At Year 5 there are somewhat more girls than 
boys. It is not clear why this should have occurred because there were no single sex 
schools in the sample. It is assumed to have arisen randomly. 

2. English as a Second Language (ESL) 

It was important to identify children for whom English is a second language because, 
according to research conducted by the English as a Second Language Unit, Western 
Australian Ministry of Education, the 'levels of competency displayed by second 
language learners do not reflect their actual levels of concept development.' ( Writing 
Developmental Continuum, p. v) If there had been a large proportion of ESL children 
used in the validation of the Continuum then the relationship between the range of 
difficulty of different phases may have been distorted. In fact there was only one 
student with English as a second language. This student was in Kindergarten. 

3. Disability affecting achievement in English. 

Teachers were asked if any of the children about whom they were making judgements 
had a disability that could 'significantly affect achievement in English'. The number of 
students described by teachers as having a disability that could affect performance in 
English was low. No kindergarten students, four Year 1 students, one Year 3 student, 
two Year 5 students and no Year 7 students were described as having a disability 
affecting achievement in English. These are small numbers so the inclusion of these 
students in the analysis is unlikely to have caused any important distortion in the 
patterns in the data. 
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4. Aboriginality 



The data identifying whether a student was an Aboriginal or a Torres Strait Islander 
were collected so that if sufficient numbers of students were identified, analyses could 
be run separately for this sub-group. There were 13 Aboriginal or Torres Strait 
Islander students in the sample. This is too few students for these analyses to be 
conducted. The data on Aboriginality were, therefore, not used in the validation of the 
Reading Development Continuum. 
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Empirical Validation of the First Steps Reading Developmental Continua 

In this part of the report four main questions are addressed: 

1. Which First Steps indicators do teachers not understand? 

2. Are the indicators within a phase at about the same level of difficulty? 

3. Do the phases reflect a sequence which implies increasing difficulty? 

4. How 'V.ey' are the 'key indicators'? 

In answering these questions, the first and fourth are addressed directly. The second 
and third questions, however, need to be approached less directly. To answer these 
questions, the method used to validate the Continuum is described. Next, an ideal 
model is proposed. This model shows how the level of difficulty of indicators within a 
phase ought to appear if a Continuum is to depict children's development. The model 
also shows how the phases ought to reflect the increasing levels of difficulty along 
such a Continuum. Once this ideal model is described, the validation of the Reading 
Development Continuum, can begin. The Continuum is evaluated by comparing 
patterns in the data with the ideal model. The closer the patterns approach the ideal 
model, the more valid the Reading Developmental Continuum will be. It is only at 
this point that the second and third questions can be answered. 

Each of the four questions is now addressed. 

Which First Steps indicators do teachers not understand? 

It is important to know how well teachers understand the indicators. Generally, if 
there are many misunderstandings then the Continuum will not be consistently used. 
More specifically, if the indicators are not well understood then the data supplied by 
teachers for the validation will be adversely effected. The data, used here to validate 
the continua, rely on teachers understanding the indicators. (Any indicators which 
teachers do not understand are excluded from the validation procedure.) 

When making judgements about a student's competency on an indicator, teachers 
were able to indicate whether they understood the indicator or not. Analysis of their 
responses shows that the indicators are well understood by the teachers in the sample. 
It should be noted, however, that this is not evidence that they all understand the 
indicators in the same way. 
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From the analysis of the TJ2' responses - those instances when teachers recorded that 
they did not understand an indicator - it was found that of the 116 indicators from the 
Reading Development Continuum 20 (17%) were not understood by at least one 
teacher. Of the 20 indicators not understood, 0 came from the 21 indicators in the Role 
Play Reading Phase, 1 came from the 26 in the Experimental Reading Phase, 4 came 
from the 27 in the Early Reading Phase, 12 came from the 23 in the Transitional 
Reading Phase and 3 came from the 19 in the Independent Reading Phase. 

Of these 20 indicators, 15 were not understood only once, 4 were not understood by 
two teachers and 1 indicator was not understood by three different teachers. The 
indicator not understood by three teachers was: "The child can discuss an alternative 
reading of a text.' (Code 2507) This indicator may require some clarification. 

Appendix C lists all those indicators from the first five phases of the Reading 
Development Continuum not understood by teachers and the number of the teachers 
who did not understand them. 

In summary, about 80% of all indicators were understood by all the teachers who 
responded to this part of the evaluation. Of the 20% of indicators not understood by 
all teachers, all but one were not understood on only one or two occasions. Thus these 
teachers seem to well understand the Reading Developmental Continuum indicators. 



Validation 



a. Calibrating Indicators 

Before considering the results of the empirical validation of the Reading 
Developmental Continuum it is necessary to outline how the data were analysed. 

The first aim of the data analysis was to Calibrate 7 the indicators for the Reading 
Development Continuum. The calibration process (based on the Rasch 3 model) 
estimates a difficulty' level for each indicator. In general, the greater the number of 
students achieving an indicator, the 'easier 7 (lower on the scale of difficulty) that 
indicator is estimated to be. In this way, the calibration process parallels the intention 
of First Steps by seeking to locate indicators at positions along a continuum. 

Named after the Danish psychometrician who invented the mathematical procedures upon 
which Item Response Theory is based. 
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Put more simply, a First Steps Continuum can be likened to a pathway along which 
children progress, acquiring literacy skills as they go. The further along this pathway 
the child goes, the more difficult it becomes. On a real path, the difficulty increases as 
a function of physical tiredness. On the metaphoric pathway to literacy, the difficulty 
increases because the behaviours the children show, require higher levels of skill. The 
First Steps indicators act as sign posts on this path. They mark out where the child is 
and so what the child has achieved (and has yet to achieve). The first aim of the 
analysis is to identify where these sign posts are along the path and, in so doing, to 
identify the difficulty of each of the indicators. 

It should be noted that this approach assumes that there is one underlying continuum 
that goes to make up the process of learning to read. Some theorists would argue that 
learning to read is a multi dimensional process which cannot, therefore, be captured 
using a single continuum. The judgement made here is that it can be captured on a 
single continuum. This judgement is based upon the view that it is reasonable to claim 
that some children read better than others. In such a claim there is an implicit 
assumption that there is only one organising dimension being tapped, so the use of a 
continuum is appropriate. 

b. An ideal model for the distribution of indicators within phases along a 
developmental continuum. 

Having estimated the difficulty of the First Steps indicators - the first aim of the data 
analysis - the next aim is to establish how well the observations that teachers make 
about the development of literacy match the developmental Continuum proposed by 
First Steps. This task is referred to as the empirical testing of the validity of the 
Continuum. 

The empirical testing of the validity of the Reading Development Continuum involves 
comparing the order of the indicators proposed by First Steps with the order produced 
by the analysis of the data supplied by teachers. This comparison, however, is 
constrained by the fact that the indicators are ordered in First Steps by allocating them 
to phases. Indicators within a phase are not ordered . 4 Consequently, if a comparison 
between the ordering of the indicators derived from the teachers' data and a First 
Steps Continuum is to be made, then the indicators must be treated by grouping them 




4 Some indicators are classified as more important (the 'Key' indicators) but they are not 
ordered in terms of the sequence of development within the phase. 
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into phases. This raises a question: If the indicators are to be ordered within phases 
how should the phases group the indicators along a scale of difficulty? 

To answer this question a model is proposed which shows the ideal relationship 
between groups of indicators within phases, when those phases are ordered to reflect 
sequential development. Such an ideal model will not be found in reality because of 
small variations in the way in which, at least some students, develop literacy skills. It 
will also not be found because of variation in the teachers' perceptions of students' 
competencies; and it will not be found because of errors made by teachers in their 
judgements. Some teachers may also use teaching strategies which direct children's 
development away from normal patterns of development. Nevertheless, such a model 
provides a standard against which to assess how well the First Steps Reading 
Development Continuum approaches the ideal. 

The model proposes, first, that the phase depicting the earliest stages of development 
should have all of its indicators situated towards the bottom or less difficult end of the 
scale. Conversely, the phase depicting the most advanced stages of development 
should have all its indicators situated towards the top or most difficult end of the scale. 
Other phases should fall between these extremes in the appropriate order. Secondly, 
each group of indicators within a phase should occupy proportionally the same 
distance along the scale of difficulty as each of the other groups of indicators. Thus, 
for example, if there are five phases each should have a range equal to 20% of the total 
range of difficulty. Thirdly each group of indicators should occupy a unique location 
on the scale. The location of the phases should not overlap along the difficulty scale. 
Exhibit 5 shows the proposed ideal distribution of indicators by phase. 

The short vertical lines in Exhibit 5 represent the spread of difficulty of the indicators 
within each phase. The most difficult indicator is situated at the top of the range of 
indicators in Phase 5. This is marked on the exhibit. Exhibit 5 shows that the spread of 
estimated difficulty for the indicators within each phase is about the same. There is no 
overlap between the phases nor is there a gap between them. 

The rationale for the characteristics of the ideal model is now given. 

Early phases should contain indicators low on the difficulty scale, intermediate phases 
should contain indicators of intermediate difficulty and later phases should contain 
indicators of highest difficulty. If the phases are not organised this way, then they are 
not depicting the sequence of development that children go through. The reason for 
having the phases is to depict this sequence of development. 
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Exhibit 5: Depiction of an ideal model showing the distribution of indicators within 
sequentially ordered phases of development. 
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There should be a spread of difficulty estimates within a phase because development 
is seen as occurring along a continuum and not in stages. If it occurred in stages it 
would be expected that each indicator within a phase would have the same estimated 
difficulty. However, the spread of indicators within a phase should not be too wide 
because this will lead to poor discrimination when plotting the development of 
children. There is little point in allocating a child to a stage if that stage, for example, 
covers a significant span of their school years. Reasonably fine levels of discrimination 
are required if development is to be charted and the indicators within a phase are to 
operate as something more than a checklist of skills. This can be achieved by having 
the phases ordered along the difficulty scale such that each occupies the same 
proportion of the total spread of the scale. It should be noted, however, that while 
'equal spread' might be thought of as an ideal, it is not necessary to the successful 
construction and use of a continuum. 

The spread of the estimates of difficulty within one phase should not overlap with the 
spread in any other phase because this can lead to difficulty in establishing the level of 
development of the child. For example, take the extreme case where the spread of 
estimated difficulty of indicators within two phases entirely overlap each other on the 
difficulty scale. When this occurs, allocating a child to one of those phases does not 
assist in locating that child along the developmental continuum. This suggests that the 
more overlap there is between phases, the more ambiguity there will be about the level 
of development of a child. 
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Ideally, gaps between the spread of estimates of difficulty within one phase and an 
adjoining phase should also not occur. A gap means that if the child is at a location in 
their development along the continuum where this gap occurs, then his or her level of 
development may be under estimated by a teacher using such a continuum. However, 
as a gap does not lead to a confusion about the sequence of development of the child 
along a continuum, it is less of a problem than having large overlaps between the 
phases. 

If the ideal model is treated as a valid depiction of how the indicators within phases 
ought to be ordered along the difficulty scale, then two of the main research questions 
can be interpreted in terms of this model. The first question - Are the indicators 
within a phase at about the same level? - requires that the spread along the difficulty 
scale of the indicators within any one phase is approximately similar to the spread of 
any other phase. The second question - Do the phases reflect a sequence which implies 
increasing difficulty? - requires that the location and overlap of the estimated difficulty 
of the indicators within each phase is compared to the ideal location of phases. This 
will involve seeing if there is any overlap or gaps between the phases and assessing 
how consequential these gaps seem to be when compared to the ideal model. 

This method of contrasting the observed with an ideal model can also be used to 
examine the location of the difficulty estimates of the key indicators within phases. 

This will permit some judgements to be made about their appropriateness as key 
indicators. The research question about their 'keyedness' is, however, largely dealt 
with by using another approach. 



C- The relationship of the ideal model of a developmental continuum to the validation 
of the First Steps Reading Development Continuum. 

The validation of the Reading Developmental Continuum will involve using the 
observations, made by teachers about the students' competencies on indicators, to 
estimate indicator difficulties. These observations can then be compared with the ideal 
model depicted in Exhibit 5. How well the teachers' observations match the model 
will shape the conclusions drawn about the validity of the Reading Developmental 
Continuum. 
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d. Validation of the Reading Developmental Continuum 

The validation begins by displaying the indicators of the Reading Developmental 
Continuum along the difficulty scale in the most detailed way. As the argument about 
the validity of the Continuum unfolds these data are displayed with less and less 
detail. It was decided to begin with the most detailed display because only this 
display allows individual indicators to be identified. Once the reader is familiar with 
this display they can refer to it if particular indicators in other displays need to be 
identified. 

Exhibit 6 shows the distribution of 99 indicators in the First Steps Reading 
Developmental Continuum along a scale of difficulty. The indicators are grouped into 
one of the five phases of this Continuum used in this validation. The Independent 
Reading Phase was not used, it will be recalled, because it was judged that no primary 
school children would be in this phase. Seven of the indicators are excluded from this 
display because an estimate of difficulty could not be calculated. The Rasch modelling 
technique requires that those indicators for which either all or none of the students 
demonstrated the behaviours described are excluded from the analysis. 5 Thus not all 
indicators are displayed in the body of Exhibit 6. The seven excluded indicators are 
listed in a box at the bottom of Exhibit 6. These indicators were excluded because all 
students demonstrated these behaviours. 

In this exhibit the indicators are represented by a code number. The code numbers 
were used to make the graphical display readable. Refer to Appendix A which lists 
these code numbers and their associated indicators. (The code numbers in Exhibit 6 
are the last two digits used in the code numbers listed in Appendix A. They identify 
the indicator. To use the Appendix, the Phase in which the indicator is located needs 
to be known.) In Exhibit 6 the key indicators are marked in bold. 



^Basically, the Rasch modelling technique estimates the difficulty of a literacy behaviour by 
comparing the number of students who exhibit a literacy behaviour with the number who do 
not. If all of the students in the sample exhibit a literacy behaviour , then it is impossible to 
estimate the difficulty of the item except to say that it is less difficult than the easiest estimated 
literacy behaviour for which there is a measure. It is not possible, however, to establish how 
much easier it is. Similarly, if none of the children demonstrate a literacy behaviour, then all 
that can be said is that it is more difficult than the hardest estimated literacy behaviour. 
Because the difficulty of these behaviours cannot be estimated, they are excluded from the 
analysis. 
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Exhibit 6: Estimated difficulty of First Steps indicators from the Reading Developmental 
Continuum within phases of development. 
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Exhibit 6 is useful for identifying individual indicators or for locating a known 
indicator. For example, it is clear from Exhibit 6 that indicator 19 in the Transitional 
Phase requires further investigation as it seems to describe a behaviour that is 
considerably easier to attain than those behaviours described by the other indicators in 
this phase. By referring to Appendix A this indicator can be identified. The text of 
indicator 19 in the Transitional Phase, is The child enjoys listening to stories'. This 
indicator is estimated to be very easy because many children exhibit this behaviour. 
This may suggest, for example, that this indicator should appear in an earlier phase. 



While detailed examination of 'outliers' such as the one discussed above can be useful 
for modifying the Continuum the present concern is with the general patterns in the 
data. To do this, less detailed displays (similar to Exhibit 5 which shows a graphical 
depiction of the ideal model) are required. These simplified versions of the data are 
now displayed and their contents examined. 

Exhibit 7: Difficulty estimates of indicators within phases of the First Steps Reading 
Developmental Continuum 
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Exhibit 7 6 shows the data as dots. An examination of this dot plot shows that there is a 
trend in these data. Generally, each successive developmental phase groups indicators 
into bands along the difficulty scale at locations which reflect increasing difficulty. 
There does appear, however, to be considerable overlap between the phases. Further 
exploration of these data is needed to clarify how serious this overlap may be. 



^The difficulty scale on this and all related plots is measured in logits. A logit is the odds of an 
outcome transformed to a logarithm with the base e. 



Exhibit 7 is unsuitable for making judgements about the spread and so the overlap of 
the phases. It is unsuitable for two reasons. First, some dots represent more than one 
indicator (those with the same estimated difficulty) and so not all data are visually 
r epresented here. Secondly, a visual examination of a dot plot such as Exhibit 7 only 
reveals the range of estimates. The range, however, is not the best measure of spread. 
It is very susceptible to the effect of outliers - points which assume an unexpectedly 
extreme value when compared with the other values in the group. There is some 
evidence in Exhibit 7 that some indicators are 'outliers'. For example, there are two 
indicators with relatively high estimates of difficulty in the Role Play Phase, another 
one which is possibly too high in the Early Reading Phase and perhaps one indicator 
with a relatively low estimate in the Transitional phase. 

There are other, better measures of the spread than the range. To complement 
Exhibit 7 then, another display summarising the data in a systematic way, which also 
reduces the effect of outlying estimates of difficulty, is required. This is done in 
Exhibit 8. Exhibit 8 shows the data using the median to represent the measure of 
central tendency and the interquartile range (the central 50% of the data around the 
median) to represent the spread. These measures are resistant to the effect of outliers. 

The data in Exhibit 8 are displayed using a type of box plot. The lower boundary of 
the box identifies the value above which 75% of the estimates fall and the upper 
boundary of the box identifies the boundary below which 75% of the estimates fall. 
Thus, 50% of the estimates are located between the top and the bottom of the box. The 
horizontal line inside the box marks the location of the median. In some boxes the 
median is not located centrally. In these cases the data are concentrated on the side of 
the box which is closer to the median. For example, in Exhibit 8, in the Transitional 
Phase, the indicators are distributed more densely towards the lower end of the box. 
The (horizontal) width of the boxes is determined by the number of indicators within a 
phase. The more indicators there are, the wider is the box. 

In examining Exhibit 8, it is the location of the median and the length and location of 
the box plot - the central 50% of the estimates - which is of most importance. It is this 
box which is taken as giving a fair representation of the spread of the indicators within 
a phase. 

The location of the median for the estimates of difficulty of each phase confirms the 
conclusion drawn from the examination of the dot plot (Exhibit 7). The median for 
each phase is located successively higher along the difficulty scale. This is consistent 
with the claim that, on average, the phases contain indicators which are grouped by 
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their location along a continuum of development. In other words, on average, phases 
contain indicators that describe increasing levels of skill or achievement. 



Exhibit 8: Box plots of difficulty estimates of indicators within phases of the First Steps 
Reading Developmental Continuum 




5 



5 - 
4 - 
3 - 
2 - 
1 - 
0 - 
-1 - 
- 2 - 
- 3 - 
- 4 - 
- 5 - 
- 6 - 



Role Play 



Experimental 



Early Reading 





Transitional Independent 



The boxes in Exhibit 8 are only about the same length. The Transitional Phase is 
noticeably shorter than the other phases. However, no one phase has a 
disproportionately wide spread of estimates. The observed data, in this regard, match 
the ideal model. This is some evidence in support of the claim that the Reading 
Developmental Continuum is a valid depiction of the order in which children acquire 
reading skills. 



The extent to which the phases within the Continuum overlap along the difficulty 
scale is now considered. An overlap between phases is a problem because if it occurs, 
to any great extent, the phases will not depict development. Its practical effect is to 
jeopardise the ability of a teacher to use the phases to track the development of a child. 
In other words, it will not be possible for teachers using the Reading Continuum to 
make meaningful judgements about the phase or level of a child a child is working at, 
unless each phase clearly represents different levels of competence. 




An examination of Exhibit 8 shows that there is very little overlap between the Early 
Reading, Transitional and the Independent Phases. There is somewhat more overlap 
between the Early Reading and the Experimental Phases. There is considerable 
overlap between the Experimental and the Role Play Phases. This is a concern 
although it is encouraging to note that the medians for these two phases are ordered 

31 



25 



appropriately. On average the Role Play Phase indicators are easier than the Early 
Reading Phase indicators. 

At this point it is important to note that the location of the median and the spread of 
data within the Role Play phase have been effected by the omission from the analysis 
of five indicators. These indicators were omitted because every child exhibited 
evidence of the behaviour that they describe. It is a requirement of the Rasch 
modelling technique that these observations are removed from the analysis because an 
accurate estimation of difficulty cannot be calculated. That all children demonstrated 
the behaviours described by these six omitted indicators means that these indicators 
are the least difficult to achieve behaviours. If they could be included in the data, their 
effect would be to lower the location of the interquartile range and probably to lower 
the median. (With the available data it is not possible to establish how much lower.) 
Consequently, the median for the Role Play Phase is probably lower than is displayed 
in Exhibit 8. Similarly, the indicators from this phase are likely to spread further down 
the scale and so not overlap as much as they do in Exhibit 8. Nevertheless, the overlap 
is considerable between these two phases. 

This finding is contrary to what had been expected. First Steps personnel reported that 
from their experience with First Steps, teachers find the later phases are harder to 
discriminate between than the developmentally earlier phases. It is not possible, with 
the data available to this study, to establish why this result is so at variance with the 
expected outcome. 

However, in First Steps the location of a child within a phase is not established by the 
use of all indicators. There are certain indicators which have been defined as 'key 7 and 
it is these Tcey 7 indicators which are used to allocate a child to a phase of development. 
Accordingly, it is how these key indicators are spread within phases and how much 
overlap there is between phases for these key indicators which is critical for 
establishing how reliably and how easily teachers can place children into a phase. 
Exhibit 9 was prepared using the estimates of difficulty for the key indicators only. 7 

Exhibit 9 shows that for the key indicators, there is a small overlap between the Early 
Reading Phase and the Transitional Phase and there is a small gap between the 
Transitional and the Independent Phases. There is a larger overlap between the 
Experimental and the Early Reading Phases. Generally, therefore, these phases are 
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In Exhibit 9 the data points and the boxes summarising the data are dsiplayed. the boxes are 
displayed to provide a consistent view of the dta. the dat points are displayed becasue there is 
so few of them, readers may wish to jhudge how adequately the boxes summarise the data. 
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roughly similar to the pattern displayed in the ideal model. The key indicators in the 
Role Play Phase, however, overlap the Experimental Phase and much of the Early 
Reading Phase. This is a concern. It suggests that the key indicators in the earliest 
phase are as difficult for children to acquire as in the following two developmental 
phases. (It should be remembered that two key indicators from the Role Play Phase 
were eliminated from the analysis because all children exhibited evidence of these 
behaviours. Had an estimate of the two missing key indicators been included with 
these data, it is likely that this phase would have less overlap with the other phases.) 



Exhibit 9: 'Box plots' with data points overlaid, of the difficulty estimates of key 
indicators within five phases of the Reading Developmental Continuum. 
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The three most difficult key indicators in the Role Play Phase, in order of difficulty, 
are; 



• The child focuses on the meaning of a story or informational text. Children's 
responses reflect understanding of the ideas presented in the text. Story 
reconstruction is meaningful. (2103) 

• The child uses prior knowledge of context and personal experience to make 
meaning, e.g. "My dog jumps up too.", 'That's like my nanna." (2105) 

• The child uses pictorial cues e.g. pointing to a picture in The Three Little Pigs', 
says 'The three little pigs left home." (2104) 



As has already been noted, this finding was unexpected. First Steps regards the Role 
Play indicators as describing very early reading behaviours. In this phase, children 
focus on the meaning of a story by using pictures or memory. It is only in the 
Experimental phase where children focus on the print and try to match spoken words 
with written words. First Steps notes the following of the Role Play Phase; 

"Children in this phase have become aware of books and the joy that 
reading brings. They have begun to notice print in the environment. 

They will display reading-like behaviours, holding the book the right 
way up and turning pages, however their reading at this stage is purely 
inventive. Children will sit turning the pages either making up a story 
from the pictures or telling the story from memory. At this stage of 
development the child is unable to match written words to spoken 
words." 

Language Development: Reading Developmental Continuum, 1992, p..4 

An examination of the wording of the two most difficult key indicators shows that this 
focus on pictures and the use of memory may not be obvious from the wording used 
in these indicators. In 2103, with the repeated emphasis on 'text' and the extraction of 
meaning from the 'text', the notion that the child is relying on pictures and memory is 
not at all clear. Similarly, 2105 does not seem to be describing an easy to acquire 
behaviour. This suggests that the wording of the indicators may require revision. In 
this context, it is interesting that the indicator which emphasises the use of pictures is 
considerably easier than the other two indicators. 

However, because this finding was so at variance with the experiences of the First 
Steps personnel, it was decided to further investigate the problem. There were four 
kindergarten teachers who supplied the data for the validation of the Reading 
Developmental Continuum. Only Kindergarten teachers provided data for the Role 
Play Phase. It was decided to further investigate these teachers' understanding of First 
Steps by examining the questionnaires which they also completed as part of the 
evaluation. 
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The four teachers came from three different schools. Of the four teachers, two 
returned questionnaires. An examination of these questionnaires showed that these 
two teachers were skilled and deliberative users of First Steps with a good grasp of the 
ideas and practices engendered by the Project. For example, one teacher wrote; 

"I found the Continua a very accurate and useful tool in helping me to 
identify where my students are 'at'. ... The information I record on the 
Developmental Continua allows me to then plan a program to advance 
the children to the next stage/ phase." 

This is a teacher who uses and understands the appropriate use of First Steps 
materials. This teacher also stresses the importance of inservicing for the proper use of 
First Steps. He or she writes; 

"It is extremely important that teachers are adequately inserviced and 
not just use some of the materials in an ad hoc fashion." 

The other Kindergarten teacher wrote that; 

'The continua provide an excellent guide on each child's development 
because each child develops at different rates and times. By using the 
continua, I have a detailed report on the level of each child at any given 
time." 

Again this suggests that this teacher is an experienced and competent user of First 
Steps materials and ideas. 

Both of these teachers used four developmental continua with their class. It seems 
unlikely that these data were provided by teachers who were unfamiliar with First 
Steps ideas or practices. It is therefore seems reasonable to have confidence in their 
judgements. This suggests that the problems with the key indicators may be with their 
wording. 

There are other features of Exhibit 9 which are also of concern. 

The experimental phase has two indicators which seem extreme. The first indicator is, 
'The child is focussed on expressing the meaning of a story rather than on reading 
words accurately.' (2202) This indicator seems, on the evidence here, to be overly 
difficult for this phase when compared with the other indicators in it. The second 




indicator is The child recognises some words and letters in context, especially letters 
from own name and other significant personal and environmental words, e.g. Stop'. 
(2214) This indicator seems to be much easier than the other key indicators in the 
Experimental Phase. 

The Early Reading Phase has one key indicator which seems to be too difficult. This is 
the indicator The child talks about ways in which people communicate ideas through 
texts such as books and advertisements for a range of purposes e.g. to entertain, to 
inform, to persuade' (2302). The Transitional Phase also has one key indicator which 
seems to be too difficult. This is the indicator The child discusses and compares own 
interpretation of texts with that of others, providing evidence relating to plot and 
characterisation in narrative or to main idea and supporting detail in informational 
text'. (2402) 

Each of these key indicators may need to be examined with a view to moving them to 
another phase or rewording them so that their meaning is clearer to teachers. 

In summary then, the evidence collected from the teachers about the behaviours 
exhibited by their students described by the First Steps key indicators suggests that the 
key indicators are, generally, grouped into the appropriate phases of the First Steps 
Reading Developmental Continuum. There is a number however, which may require 
further investigation. The developmentally earliest phase - the Role Play Phase - 
seems to have key indicators which are very difficult compared with the indicators in 
the developmentally later Experimental and Early Reading Phases. 

Finally, it should be noted that where there are large overlaps between phases or 
where a developmentally later phase is on average less difficult than an earlier phase 
there are a number of possible reasons for this occurring. Some of these reasons 
include: 

• Some indicators have been placed in the wrong phase by First Steps. By moving the 
incorrectly located indicators into the correct phase the problems of overlap 
and of incorrect sequencing of the phases will be remedied. 

• Some indicators are imprecisely worded leading teachers to have differing 
interpretations of them. If the wording of these indicators is changed to clarify 
their meaning then the indicators may be more precisely located. 

• Some examples used to illustrate a behaviour, which form part of the wording of an 
indicator, may be misleading. A review of the effect of the examples on the 
interpretation of the meaning of an indicator could lead to a more precise 
interpretation of the indicator by teachers. 
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• Some phases are intrinsically difficult to discriminate between. It may be the case 
that the differences between two phases are both real and consequential but 
also subtle and difficult to detect. In this case, teachers' interpretations will be 
inconsistent when attempting to distinguish between phases. 

• Some pedagogical practices may discourage children from folloxving the development 
depicted in the Continua. 

Each of these factors may alone or in conjunction affect the estimated difficulty of the 
indicators. 



e. How 'key' are the key indicators? 

Every phase of a First Steps Continuum has some indicators defined as 'key'. These 
indicators are used to allocate children to a phase of development. They were defined 
as 'key' by First Steps using in-house research done by First Steps personnel and 
advice taken from the users of First Steps. 

The aim of this section is to examine how Tcey' these 'key indicators' are. First, 
however, an introduction is given to the method by which the Tceyedness' of the key 
indicators is estimated. 

Earlier, a First Steps Continuum was likened to a path along which indicators are 
placed to act like milestones or sign posts. These milestones tell the teacher where the 
child is in their development. So far, the analysis has been concerned with locating 
where these milestones are along the length of the path. The next task might be 
likened to establishing how close these milestones are placed to this path. If the 
milestones are close to the path then their message is clear and unambiguous about a 
child's location on the path. Such indicators would be good candidates for being 
defined as key indicators. The further from the path the milestones lie, the more 
indistinct and ambiguous becomes their message. If they are a long way from the path 
it may not be clear that they refer to this path at all but to another. 

This analogy, using milestones and paths needs to be treated cautiously for, like all 
analogies, it is limited. A more precise description of the approach used is needed. A 
First Steps Continuum can be treated as a variable. This variable, when measured can 
be construed as tapping an underlying 'trait' named, say, 'reading ability' (in the case 
of the Reading Developmental Continuum). Once conceived this way, the Rasch 
modelling technique can be used to provide an estimate of the extent to which any one 
indicator is consistent with this underlying variable. If a 'key indicator' is defined as 
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an indicator which is strongly consistent with the underlying variable, then Rasch 
modelling can be used to estimate how 'key' the key indicators of a First Steps 
Continuum are. It can also show which of those indicators that are currently not 
defined as 'key', might be good candidates for being defined as such, if changes are 
required. 

The measure used to estimate the consistency of an indicator with the underlying 
variable is called the Tnfit Mean Square'. Exhibit 10 shows values of the Infit Mean 
Square for each of the indicators from the Reading Developmental Continuum. The 
scale has been divided into three zones. There is a left hand zone, a central zone which 
has a column of dots marking its boundaries and a right hand zone. Although the 
locations of these zones are somewhat arbitrary they are based upon practical 
experience with this technique. 8 

An indicator with a value falling inside the central zone is measuring the underlying 
variable to a satisfactory extent. In Exhibit 10 an example of one such indicator is the 
First Steps indicator coded with the number 2112. 

An indicator with a value in the right hand zone (that is with an Infit Mean Square 

t 

value of about 1.4 or more) has a less than ideal correlation with the other indicators 
developed for this Continuum. (See Indicator 2104, for example, in Exhibit 10.) 

Indicators with Infit Mean Square values in the left hand zone of Exhibit 10 are 
strongly correlated with the dimension being defined by the full set of indicators. 

These indicators can be regarded as candidates for being defined as 'key indicators'. 
Indicator 2102 in Exhibit 10 is an example of one such indicator. 

In Exhibit 10 the letter K' in the body of the table marks the location of the Infit Mean 
Square for each of the key indicators of the Reading Developmental Continuum. The 
Infit mean squares of all other indicators are marked using an aster '*'. The first 

column lists the code number of each of the indicators in the Reading Developmental 
Continuum that were included in the analysis. Appendix A provides the link between 
these code numbers and the text of the indicators. (It might be useful, however, to 
note here that the second digit of the code number identifies the phase of the indicator. 
Thus the digit '1' in this location means the first or the 'Role Play Phase' and '2' means 
the 'Experimental Phase' and so on.) 
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®Given the use of the 'path' as an analogy for a First Steps continuum throughout this report it 
is probably important to note that the central column in Exhibit 10 Ought not to be regarded as 
representing a path. 
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Exhibit 10: Item fit of First Steps Reading Developmental Continuum indicators. 



O 
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First Steps 

Indicator 

Code 



2102 

2103 

2104 

2105 

2106 

2107 

2108 
2109 
2111 
2112 

2115 

2116 

2117 

2118 
2119 
2201 
2202 

2203 

2204 

2205 

2206 

2207 

2208 

2209 

2210 
2211 
2212 

2213 

2214 

2215 

2216 
2218 

2219 

2220 
2221 
2222 

2223 

2224 

2225 

2226 

2301 

2302 

2303 

2304 

2305 

2306 

2307 

2308 

2309 

2310 

2311 

2312 

2313 

2314 

2315 

2316 

2317 

2318 

2319 

2320 



Infit Mean Square 

0.45 0.56 0.71 1.00 

— + + + +- 

K 

K 



1.40 1.80 2.20 

■ — + + +- 



K 



K 



K 

K 



★ 



★ 



K 



★ 



K 



★ 



★ 



★ 



K 



K 



★ 



★ 



★ 



K 



K 



★ 



★ 



★ 




★ 

★ 

Continued next page 
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Exhibit 20: Item fit of First Steps Reading Developmental Continuum indicators (Continued) 



First Steps 
Indicator 

Code 0.45 



2321 

2322 

2323 

2324 

2325 

2326 

2327 

2401 

2402 

2403 

2404 

2405 

2406 

2407 

2408 

2409 

2410 

2411 

2412 

2413 

2414 

2415 

2416 

2417 

2418 

2419 

2420 

2421 

2422 

2423 

2501 

2502 

2503 

2504 

2505 

2506 

2507 

2508 

2509 

2510 

2511 

2512 

2513 

2514 

2515 

2516 

2517 

2518 

2519 



Infit Mean Square 

0.56 0.71 1.00 1.40 1.80 2.20 

■ — + + + + + +- 

★ 

K 

K 



K 



K 



K 



K 



K 

K 



K 

K 



* 



K 



K 



★ 



K 



K 



K 



K 



★ 



* 



K 



* 



* 



★ 



★ 



K 



★ 



K 



An examination of Exhibit 10 shows that most key indicators in the Reading 
Developmental Continuum fall within the central zone or the left hand zone. These 
key indicators can thus be considered consistent with the concept that underlies or 
defines this Continuum, namely 'reading ability'. In the vernacular - these key 
indicators all 'hang together well'. 
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There are, however, a number of key indicators which do not seem to be contributing 
to the variable underlying the Reading Development Continuum. There are two key 
indicators in the Role Play Phase. These are The child uses pictorial cues e.g. pointing 
to a picture in The Three Little Pigs', says "The three little pigs left home." (2104) and 
The child uses prior knowledge of context and personal experience to make meaning, 
e.g. "My dog jumps up too.", "That's like my nanna." (2105) Of these two, the first - 
2104 - is the more disturbing. It has quite an extreme Infit Mean Square value, 
suggesting that this indicator needs careful consideration. 



The other three key indicators which do not seem to be contributing to the underlying 
variable defining the Continuum are 2304, 2404 and 2515. These are not as extreme as 
2104 but should, nevertheless, be closely examined to clarify their meaning, or to 
remove them from the list of indicators used to describe the Reading Development 
Continuum. 

Those indicators which are currently not defined as key but might be considered as 
candidates for being defined as key, on the basis of these data, are as follows: 



Phase 

Role Play 
Experimental 
Early Reading 
Transitional 
Advanced 



Indicator code 

2106, 2107,2108, 2109, 

2207, 2208, 2213, 2216, 2218, 2221 
2306, 2316, 2317, 2326, 2327 
nil 

2506, 2512 



The indicators from the Reading Developmental Continuum which are identified as 
having a relatively low correlation with the majority of the indicators are: 



Phase 

Role Play 
Experimental 
Early Reading 
Transitional 
Independent 



Indicator code 

21 04 (key), 2105 (key), 2111 , 21 1 8, 21 1 9 
2222, 2223, 2224, 2225 
2304 (key), 2311, 2312, 2313, 2321, 2324, 
2404 (key) 

2513, 2515 



There are thus 18 indicators in the Reading Developmental Continuum which could be 
considered for either revision of their wording or removal from the Continuum 
because, on the data used here, they are not consistent or strongly consistent with the 
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Continuum of which they have been defined as part. More positively, about 80% of 
indicators are either consistent or are strongly consistent with the continuum 

There is some evidence in Exhibit 10 that the phases describing behaviours that occur 
late in children's development are more coherent than the earlier phases. In particular, 
the Role Play Phase seems to require reworking. This is especially so given that two 
key indicators from this Phase have a high (therefore, poor) Infit Mean Square. 
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PART 2: THE RESPONSE OF CLASSROOM TEACHERS TO THE FIRST 
STEPS CONTINUA 



In this section of the report the views held by Kindergarten, Year 1, 3, 5 and 7 
classroom teachers about the usefulness and the validity of the First Steps Continua 
are examined. These views are taken from responses to questionnaires. The 
questionnaires were sent to selected schools in November 1993. The empirical 
validation in the first section of this report only examined the Reading Developmental 
Continuum. In this section, because teachers were asked about the five First Steps 
Continua, all continua are considered. 



There were 157 teachers from 31 schools who agreed to participate in this part of the 
evaluation. 9 Responses came from 120 classroom teachers in 26 schools. This 
represents a response rate from teachers of around 75%. Of these 120 teachers, 16 were 
teaching Kindergarten at the time of the survey, 22 were teaching Year 1, 15 were 
teaching Year 3, 14 were teaching Year 5, 14 were teaching Year 7, 33 were teaching 
combined Year level classes and 5 were teaching either Year 2, 4 or 6. It is not clear 
how the Year 2, 4 or 6 teachers came to answer the survey because these Year level 
teachers were not asked to participate. Their responses have been included. This is 
justified on the grounds that, as the sample was not randomly drawn, little was to be 
lost by including them, and perhaps much was to be gained. 

The sample was not randomly drawn and the findings from this part of the evaluation 
are not genera lisa ble. They can only be used either as pointers to further research, or 
to gain insights, or to draw on the wealth of experience that teachers have had with 
First Steps. 



Examination of some survey data about the First Steps Continua from classroom 
teachers. 




Teachers were asked to identify the First Steps Continua which they had used in their 
class in 1993. The most commonly used was the Spelling Developmental Continuum 
(85% of the teachers), followed closely by the Writing Developmental Continuum 
(80%). The Writing Learning Continuum, the Reading Developmental Continuum and 
the Oral Language Developmental Continuum were used by between 35% and 40% of 

^These were the teachers and schools who supplied data for either the validation of the 
Reading Developmental Continuum or who provided data for the revalidation of the Spelling 
Developmental Continuum. Their questionnaire responses are combined in this section of the 
report. This is why, for example, in Part 1 there are only four Kindergarten teachers who are 
reported as supplying data whereas i this part of the report there are 16 Kindergarten teachers. 
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the teachers. As a consequence, most of the general comments they make (that is 
when they do not refer explicitly to a specific Continuum) probably refer to the 
Spelling and Writing Developmental Continua. 

Only 13 teachers reported using only one Continuum in 1993. Most reported using 
two Continua. There were 44 (or 37% of teachers) using two Continua. About a 
quarter of teachers reported using three or four Continua in their class during 1993. A 
small number, 4, reported using all five Continua. Most teachers replying to the 
survey, therefore, have had wide experience with First Steps. This makes their 
comments important. 

So, to what extent do teachers in this sample regard the Continua as accurately 
depicting the development of skills in the students which they teach? 

Teachers were asked in the survey to describe how well the Continua which they had 
used, depicted the development observed in their students. Some teachers 
commented upon the Continua in general, and others referred to a specific continuum. 
Their responses were coded into 3 broad categories. Exhibit 10 describes the 
frequencies in each of these categories. 



Exhibit 25: Number and percentage of classroom teachers reporting how well the 
Continua which they used in 1993, depict the development of their students. 



Continua 


Very Well 


Reasonably 

Well 


Poorly 


No Comment 
or Missing 


Writing Development 


14 


5 


0 


101 


Writing Learning 


4 


0 


0 


116 


Spelling Development 


16 


6 


1 


97 


Reading Development 


2 


1 


0 


117 


Oral Language Development 


2 


2 


0 


116 


Continuum not identified 


65 


17 


2 


36 



Note that as there is double counting down the columns, no column totals are provided. All 
rows total to 120. 



These data show that, generally, so far as these classroom teachers are concerned, the 
Continua accurately depict the development of literacy skills of their students. 
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Some comments from the teachers will help to give a sense of how they see the 
Continua as depicting development and the implications this has for their teaching 
practice. 

A number of teachers did not get to specifics, but did comment very positively about 
the continua. One teacher wrote, for example; 

"The Continua depict the development of the students particularly well. 

The development/ progression of each child can be seen clearly. By 
using the continua the classroom teacher has a base to work from 
[using] the clearly set out strategies." 

Of those 65 teachers who, without specifying Continua, reported that the Continua 
depicted the development of children very well, 25 of them referred to the strategies 
that First Steps provided in conjunction with the Continua. For example, one teacher 
wrote; 



"Very good! It gives you an accurate picture of where the children are, 
and from there you've got the appropriate strategies/activities to move 
them along to the next phase. For example my class fell down on 
editing or punctuation skills at the beginning, and the Writing 
Continuum picked this up .." 

Other teachers commented positively about how accurate the indicators are. Others 
referred to how helpful the Continua are for communicating with parents. Others 
focused upon the effects on the children. One teacher, for example, wrote; 

'The preliminary spelling phase reflects very accurately, the 
development seen in our Pre-Primary (5 year old) children. I have not 
gone 'overboard' this year with trying to implement First Steps but by 
following the 'Major Teaching Emphases' for the Preliminary stage [I 
feel] that this group of children are more aware of print and the purpose 
and actual mechanics of writing, than previous groups have been. Even 
those not yet interested in recording are fully aware of the differences 
between print and pictures." 

Some teachers were less enthusiastic. These were defined as viewing the Continua as 
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depicting 'Reasonably Well' the development of the children in their class. A typical 
example is as follows; 

'They [the Continual depict the development of most students quite 
well - as anticipated by our own observations.' 

Others were more specific. About half of the teachers who felt the Continua showed 
the development of their children only reasonably well referred to ambiguity in the 
wording of the indicators or to other practical problems. For example, one teacher 
wrote; 



"Very accurate examples on continua help to establish the phase at 
which each child is. However, [for] those children who are not yet [at] 
any phase it is difficult to assess their development until they reach a 
phase." 

Some teachers also complained that the continua were too detailed. Others thought 
the opposite. One argued; 

'The movement on the continua is far too slow. [N]ot enough progress 
can be seen. There need to be more indicators to show improvement." 



Others were more specific. 

"I feel the students are more willing to have a go, however, I don't think 
it does anything to remedy the really weak children. With some who 
have error after error, their spelling journals are filled to the limit with 
no real sense of achievement at the end." 



The teachers were asked to comment upon how the Continua could be further 
improved and developed. Their responses are summarised in Exhibit 11. This shows 
that the most commonly reported problem with the continua is ambiguity in the 
wording of the indicators. A typical example of such a concern comes from a teacher 
who wrote that; 
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"Some comments are a trifle hard to interpret, e.g. 'Willing to have a go 
at representing speech in print.' Needs clarifying - some people argue 
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that it just means spoken words others thing children need to do actual 
"[quote marks]"." 

Others are less specific. For example, 

"Some parts of each continua are open to individual interpretation 
which may cause difficulties as the continua are passed through the year 
levels.' 

Exhibit 11: Teachers' views of how the First Steps Continua could be improved. 



Reason 


N. 


Percent 


No changes needed 


13 


11 


Phases too broad 


13 


11 


Indicators too detailed 


13 


11 


Simplify terminology 


4 


3 


Ambiguity of wording 


37 


31 


No negative indicators 


5 


4 


More materials needed 


11 


9 


Other 


9 


8 


Missing 


13 


11 


Total 


120 


100 



Note that only the most important reason from each response try a teacher was coded. There is 
no double counting in Exhibit 11. The most important reason was the one identified as such by 
the respondent or which had the most words given up to it by the respondent or, where 
neither of these applied, was the first reason given. 

Finally, teachers were asked to make a general assessment of how helpful each of the 
First Steps Continua had been to them through out 1993. Exhibit 12 shows that for 
most teachers responding to this item, the Continua were either 'Very Helpful' or 
'Helpful'. A small number reported that the continua were neither helpful nor 
unhelpful. Even fewer reported that the Continua were unhelpful. 
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Despite the reservations that teachers express about some aspects of the First Steps 
Continua, then, the group in the sample used here, are overwhelmingly of the view 
that the Continua are helpful to them. This can be seen most clearly with the Spelling 
Developmental Continuum where 88% of teachers in this sample reported the 
Continuum as either helpful or very helpful. It needs to be noted at this point that 
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because of the sampling method used to select teachers, these findings, are not 
generalisable. They do, however, co-incide with, generalisable findings from the 
earlier ACER studies of First Steps. 



Exhibit 12: Number of teachers reporting how helpful the First Steps Continua had 
been to them in teaching their class in 1993. 



Continua 


Very 

Helpful 


Helpful 


Neither 


Unhelpful 


Very 

Unhelpful 


Total 

response 


Writing 

Development 


46 

(47%) 


44 

(45%) 


7 

(7%) 


0 


0 


97 

(100%) 


Writing Learning 


23 

(55%) 


18 

(43%) 


1 

(2%) 


0 


0 


42 

(100% 


Spelling 


49 

(48%) 


48 

(47%) 


4 

(4%) 


1 

(1%) 


0 


102 

(100% 


Reading 


22 

(43%) 


24 

(47%) 


4 

(8%) 


1 

(2%) 


0 


51 

(100% 


Oral Language 


9 

(27%) 


19 

(58%) 


4 

(12%) 


1 

(3%) 


0 


33 

(100% 



In summary then, teachers in the sample used here, find the First Steps Continua to be 
helpful, to depict the development of the children in their class with accuracy and they 
claim this despite some concerns with the wording of some of the indicators in the 
various Continua. 
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APPENDIX A: 



Code numbers for the First Steps Reading Developmental Continua indicators 

For ease of use and to avoid cluttered graphical displays, indicators were allocated 
codes. The codes and the indicators are listed below. 



The first digit identifies the Continuum from which the indicator comes. (This is 
consistent with the numbering of codes used in other publication associated with 
the ACER evaluation of First Steps. Strictly, this code number is unnecessary in 
this report.) The second digit identifies the phase in which the indicator is 
located. For this second digit the code 1 = the Role Play Phase, 2 = the 
Experimental Phase, 3 = Early Reading Phase, 4 = Transitional Phase, 

5 = Independent Phase and 6 = Advanced Phase. (The Advanced Phase is not 
used in the validation. The Advanced Phase indicators are included here for 
completeness.) The third and fourth digits identify the indicators. The fifth 
character of the code indicates the response set available to teachers when they 
were entering the data. For those indicators with the code 'B' the teachers had 
the 'Yes' / 'No' / 'Beyond' response categories and for the code T' they had the 
"Yes' / 'No' response categories. 



1. Role Play Phase, Reading Continuum 

2101 Y 

displays reading-like behaviour 

- holding book the right way up 

- turning the pages appropriately 

- looking at words and pictures 

- using pictures to construct ideas 

2102 B 

realises that print carries a message but may read the writing differently each 
time e.g. when reading scribble to parents 

2103 Y 

focuses on the meaning of a story or informational text. Children's responses 
reflect understanding of the ideas presented in the text. Story reconstruction 
is meaningful. 

2104 Y 

uses pictorial cues e.g. pointing to a picture in The Three Little Pigs', says 
'The three little pigs left home." 

2105 Y 

uses prior knowledge of context and personal experience to make meaning, 
e.g. "My dog jumps up too.", 'That's like my nanna." 



Role Play Phase, Reading Continuum (Cont.) 

2106 B 

uses pictorial cues to help reconstruct the story 

2107 B 

turns the pages of the book, telling the story from memory 

2108 Y 

knows that writing and drawing are different e.g. "Mummy reads the black 
bits." 

2109 Y 

reacts to environmental print e.g. noticing a fast food sign the child says, "I 
want a hamburger." 

2110 Y 

selects favourite books from a range e.g. selects a book saying, "I want The 
Three Little Pigs." 

2111 Y ' 

is familiar with and talks about stories 

2112 B 

is beginning to use some book language appropriately, e.g. 'Once upon a time 
...'. The child may use a 'reading voice'. 

2113 Y 

recognises own name or part of it in print 

2114 B 

is beginning to recognise some letters e.g. Sam says "That's my name", 
pointing to Stop sign. 

2115 B 

displays curiosity about print by experimenting with 'writing' and drawing 
and asking, "What does that say?" 

2116 Y 

wants to look at books 

2117 B 

offers to 'read' writing and points to text while 'reading', indicating the 
beginning of 'having-a-go' 

2118 Y 

enjoys stories and asks for them to be read and re-read 

2119 Y 

spontaneously talks about favourite part of stories 



Appendix A: Code Numbers for the Reading Continuum Indicators 



Role Play Phase, Reading Continuum (Cont.) 

2120 B 

expresses enjoyment by dapping, joining in orally and responding emotively 
when listening to familiar stories. 

2121 Y 

eagerly anticipates book-reading events that are part of daily routine 

2. Experimental Phase, Reading Continuum 
2201 

realises that print contains a constant message, e.g. that the words of a 
written story remain the same, but the words of an oral story may change 

2202 Y 

is focussed on expressing the meaning of a story rather than on reading 
words accurately. 

2203 Y 

uses prior knowledge of context and personal experience to make meaning, 
e.g. uses memory of a text to match spoken with written words 

2204 Y 

knows that print goes from left to right and from top to bottom of a page 

2205 B 

responds to and uses simple terminology such as book, right way up, front, 
back, upside down, letter, word, sentence. 

2206 B 

is beginning to demonstrate awareness of literary language, e.g. 'a long, long 
time ago ...' 'by the fire sat the cat', '"No, no, no", said the ...'. 

2207 Y 

can tell what a story is about from the title and illustrations, e.g. "I want the 
story about the big black cat" 

2208 B 

shows an ability to connect ideas and events from stories by retelling events 
in a story in sequence using pictures, memory of the story and knowledge of 
story structure. 

2209 Y 

expresses personal views about the actions of a character and speculates on 
own behaviour in a similar situation, e.g. "If I had been ... I would have ..." 

2210 B 

sub vocalises or whispers when reading silently 
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Appendix A: Code Numbers for the Reading Continuum Indicators 



Experimental Phase , Reading Continuum (Cont.) 

2211 B 

is beginning to demonstrate understanding of one-to-one correspondence 
between spoken and written words, for instance the child slows down when 
dictating to an adult 

2212 Y 

asks for assistance with some words. May be aware that own reading is not 
accurate and may seek help, re-read or stop reading 

2213 Y 

uses patterns of language to predict words or phrases 

2214 B 

recognises some words and letters in context, especially letters from own 
name and other significant personal and environmental words, e.g. Stop 

2215 B 

is beginning to match some spoken words with written words when reading 
a book or environmental print 

2216 B 

is developing the ability to separate a word from the object it represents. For 
instance the child realises that 'Dad' is a little word not that, 'Dad' is a big 
word because Dad is big. 

2217 B 

recognises some letters of the alphabet and is able to name them 

2218 B 

demonstrates some knowledge of letter-sound relationships, for instance, 
the sound represented by the initial and most salient letters in words. 

2219 B 

points to specific known words as they are read 

2220 Y 

uses initial letter sounds to predict words in text 

2221 B 

is beginning to see self as a reader and talks about own reading 

2222 B 

enjoys stories and asks for them to be read and re read 

2223 v 

takes risks by guessing and inventing meanings 

2224 B 

joins in and acts out familiar stories 
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Experimental Phase, Reading Continuum (Cont.) 

2225 Y 

eagerly selects books to read for pleasure 

2226 Y 

self-selects texts on the basis of interest or familiarity 



3. Early Reading Fhase 

2301 Y 

carefully reads text, demonstrating the understanding that meaning is vested 
in the words 

2302 Y 

talks about ways in which people communicate ideas through texts such as 
books and advertisements for a range of purposes e.g. to entertain, to inform, 
to persuade 

2303 Y 

can identify and talk about a range of different text forms, such as letters, 
lists, recipes, stories, informational texts 

2304 Y 

retells the main content of a story or informational text and supplies some 
supporting information 

2305 Y 

looks for and talks about ideas and information from informational texts, 
making links to own knowledge 

2306 B 

is beginning to read familiar texts confidently, e.g. language experience 
recounts, shared books 

2307 Y 

elaborates answers to questions, e.g. 'It is a scary story because it's about a 
mean old witch who eats people.' 

2308 Y 

makes comparisons with other texts read. The child's comments could relate 
to theme, character, plot, information 

2309 Y 

talks about characters in books using picture clues, personal experience and 
knowledge of the text to make inferences 

2310 Y 

provides detail about characters, setting and events when retelling a story 
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Appendix A: Code Numbers for the Reading Continuum Indicators 



Early Reading Phase, (Coni.) 



2311 Y 

interprets advertisements and ideas and information from texts from 
personal viewpoint 

2312 Y 

when questioned can reflect on own strategies at the level of the whole text, 
e.g. "I knew it was going to be about a baby dinosaur 'cos I looked at all the 
pictures first." 

2313 Y 

reads silently, but may sometimes sub vocalise 

2314 B 

may read word-by-word or line-by-line when reading an unfamiliar text, i.e. 
reading performance may be word centred. Fluency and expression may 
become stilted as the child focuses on decoding. 

2315 B 

generally makes meaningful substitutions, however over reliance on 
grapho-phonics may cause some meaning to be lost 

2316 Y 

uses picture cues, knowledge of language patterns and context to work out 
the meaning of unknown words 

2317 B 

is beginning to use self-correction as a strategy 

2318 Y 

uses knowledge of sentence structure and punctuation to help make 
meaning (syntactic strategies) 

2319 B 

sometimes reads-on to confirm meaning, however some children may read 
on regardless 

2320 Y 

re-reads passage in order to clarify meaning that may have been lost due to 
word-by-word reading. May re-read a phrase, a sentence or a paragraph 

2321 Y 

when questioned can reflect on own strategies at the sentence level e.g. 'I 
knew he was going to say that, 'cos he said it two times already.' 

2322 Y 

has a bank of words which are recognised when encountered in different 
contexts, e.g. in a book, on the blackboard or on a chart 

2323 B 

relies heavily on beginning letters and sounding out for word identification 
(graphophonic) strategies 
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Early Reading Phase, (Cont.) 



2324 B 

points as an aid to reading, using finger, eyes or voice 

2325 B 

locates words from sources such as word banks and environmental print 

2326 Y 

when questioned can reflect on own word identification strategies, e.g. "I 
sounded it out." 

2327 Y 

is willing to 'have-a-go' at reading unknown words 
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4. Transitional Phase, Reading Continuum 

2401 Y 

shows an ability to construct meaning by integrating knowledge of: 

- personal experiences 

- text structure, e.g. a letter, a recount, a report, a narrative 

- text organisation, e.g. introduction, conclusion, paragraphs, chapters 

- language features, e.g. narrative - descriptive language, reports - subject 
specific vocabulary, explanation - cause/effect relationships 

- context and content 

2402 Y 

discusses and compares own interpretation of texts with that of others, 
providing evidence relating to plot and characterisation in narrative or to 
main idea and supporting detail in informational text 

2403 Y 

uses a range of strategies effectively to find relevant information in texts, 
e.g. makes use of table of contents and index 

2404 Y 

recognises that characters can be stereotypes in a text, e.g. a mother looking 
after children at home while the father goes to work, a prince rescuing a 
maiden from a wicked stepmother and suggests how this could be changed. 

2405 Y 

makes inferences and predictions based on information which is explicitly 
stated in a text 

2406 Y 

makes generalisations based on interpretation of texts viewed or read, e.g. 
confirms, extends, or amends own prior knowledge through reading or 
viewing 

2407 Y 

adjusts reading strategies for different texts and different purposes, e.g. 
searching for a specific fact in an informational text 
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Transitional Phase, Reading Continuum (Cont.) 

2408 B 

reads orally with increasing fluency and expression. Oral reading reflects 
personal interpretation 

2409 Y 

makes comparisons with other texts read 

2410 B 

is becoming efficient in using most of the following strategies for 
constructing meaning: 

- makes predictions and is able to substantiate them 

- self-corrects when reading 

- re-reads to clarify meaning 

- reads on when encountering a difficult text 

- slows down when reading difficult texts 

- substitutes familiar words 

- uses knowledge of print conventions 

2411 B 

makes meaningful substitutions, i.e. replacement miscues are meaningful, 
e.g. 'cool' drink for 'cold' drink. The integration of all three cueing systems 
(semantic, syntactic and grapho-phonic) are developing 

2412 Y 

uses more print conventions as an aid to understanding, e.g. capitalisation, 
full stops, commas, exclamation marks, speech marks 

2413 Y 

is able to reflect on and describe some of the strategies for making meaning 

2414 Y 

has an increasing bank of sight words, including some difficult and subject- 
specific words, e.g. science, experiment, February, Christmas 

2415 B 

is becoming efficient in the use of the following word identification 
strategies when appropriate: 

- sounds out to decode words 

- uses initial letters as a cue to decoding 

- uses knowledge of common letter patterns to decode words, 
e.g. th, tion, ough 

- uses known parts of words to make sense of the whole word 

- uses blending to decode words, e.g. str-ing 

- uses word segmentation to make sense of the whole word 

2416 Y 

is self-motivated to read for pleasure 

2417 Y 

reads for a range of purposes 
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Transitional Phase, Reading Continuum (Cont.) 

2418 Y 

selects texts effectively, integrating reading purpose and level of difficulty 

2419 Y 

enjoys listening to stories 

2420 Y 

reads for a range of purposes, e.g. for pleasure or information 

2421 Y 

responds sensitively to stories read 

2422 Y 

discusses favourite books and talks about favourite author 

2423 Y 

selects own reading material according to interest, perceived purpose and 
level of difficulty 



5. Independent Phase, Reading Continuum 

2501 Y 

reads and comprehends text that is abstract and removed from personal 
experience and can justify own interpretation of the text 

2502 Y 

makes inferences based on information implicit in a text and can provide 
supporting evidence, e.g. having read material on diet and beauty infers that 
being slim is valued by society and that this is evidenced by the huge sales of 
alternative fad diets 

2503 Y 

returns purposefully to make connections between widely separated sections 
of a text 

2504 Y 

makes critical comparisons between texts 

2505 Y 

applies basic research skills effectively such as identifying informational 
needs, e.g. knowledge of library organisation, knowledge of text 
organisation, and extracting relevant information 

2506 Y 

talks with others about interesting or difficult content 

2507 Y 

can discuss an alternative reading of a text 
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Independent Phase, Reading Continuum (Cont.) 

2508 Y 

can justify own interpretation of a text and offer possible reasons why a text 
may interpreted differently by different people 

2509 Y 

comments and makes judgements on the ways authors represent people 
from different cultural and socio-economic groups 

2510 B 

is beginning to recognise and appreciate that authors manipulate language 
a variety of ways to clarify and enhance meaning 

2511 Y 

can recognise and discuss the elements and purpose of different text 
structures, e.g. biography, mystery in fiction; exposition, procedure 

2512 Y 

reflects on and discusses issues and topics that have emerged when reading 

- challenges and criticises text, offering supportive evidence 

- organises logical responses to a text 

- selects relevant information for own purpose 

- identifies and synthesises points of view 

- draws conclusions from texts and generalises information extracted from 
them 

- generates expectations about plot and character development 

2513 Y 

may draw analogies from fiction to gain heightened awareness of self 

2514 Y 

reads and comments critically on news items, magazine articles, 
advertisements and letters in the press 

2515 Y 

uses a range of strategies automatically when constructing meaning from 
text: 

- self corrects 

- rereads 

- reads on 

- slows down 

- sub vocalises 

2516 Y 

uses word identification strategies appropriately and automatically when 
encountering an unknown word 

- knowledge of grapho-phonics 

- knowledge of word patterns 

- knowledge of word derivations and root words 

- syllabification 




Appendix A: Code Numbers for the Reading Continuum Indicators 



Independent Phase, Reading Continuum (Cont.) 

2517 Y 

may discover a particular genre, e.g. horse books or adventure stories and 
seek out other texts of this type 

2518 Y 

makes comparisons between texts 

2519 Y 

demonstrates confidence when reading 



6. Advanced Phase, Reading Continuum 

2601 Y 

can critically reflect on and respond to text providing different levels of 
interpretation and adopting alternative view points 

2602 Y 

can stand back and reflect on own reactions to author's perceived ideologies 
and positions 

2603 Y 

can recognise specific language forms such as figurative language, jargon and 
technical language 

2604 Y 

can recognise and describe the purpose and structure of different genres 

2605 Y 

reflects personal interpretation of the text through oral reading 

2606 Y 

can recognise and respond to text complexity, e.g. ambiguity and conflicting 
messages in text 

2607 Y 

can compare and contrast different points of view 

2608 Y 

can identify and integrate layers of facts and concepts within a text 

2609 Y 

can identify and discuss different authors' styles 

2610 Y 

can recognise texts as 'cultural constructs ' and can analyse the cultural beliefs 
underpinning texts 

2611 Y 

can synthesise information within and across texts 
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Advanced Phase, Reading Continuum (Cont.) 

2612 Y 

is able to select, use, monitor and reflect on appropriate strategies for 
different reading purposes, 

2613 Y 

is able to interrogate text, articulating problems and formulating relevant 
questions 

2614 Y 

can select key information and ignore irrelevant material 

2615 Y 

can apply understanding of text structure to the acquisition, organisation and 
application of information 

2616 Y 

can formulate and apply research strategies 

2617 Y 

can recognise and analyse bias, propaganda and stereotyping in texts 

2618 Y 

can draw on literary and cultural repertoires to construct meanings in order 
to compare the perceived world view of an author with their own 

2619 Y 

uses reading to enter worlds beyond personal experience 

2620 Y 

confidently handles new text 

2621 Y 

responds sensitively and perceptively to literature 

2622 Y 

reads literature with emotional involvement and reflective detachment 
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APPENDIX B: 



Estimates of the difficulty of the indicators from the First Steps Reading 
Developmental Continuum 

The measure of difficulty of an indicator was estimated using Item Response 
Theory. This technique requires that those indicators for which either all or 
none of the students demonstrated a behaviour are removed from the analysis. 
Consequently, not all indicators appear in the lists below. The estimate of 
difficulty of the indicators is measured by using the natural log odds of the 
proportion of students who were identified as not having demonstrated those 
behaviours described by an indicator. In this appendix, the estimates of difficulty 
are first ordered by the indicator code number and then ordered by the estimated 
difficulty of the indicator. 

A note on the indicator codes: 

The indicator codes have four digits. The codes are the same as those used in 
Appendix A. The response set codes 'B' and / Y', used in Appendix A, are not 
used here. Appendix A will need to be used to link the estimates shown here 
(Appendix B) back to the wording of the indicators. 

The code numbers are ordered so that the first digit identifies the Reading 
Developmental Continuum, the second digit identifies the phase and the third 
and fourth digits identify the indicator. The phase code numbers are in the order 
used by First Steps and so reflect the sequence that they have in the continuum. 
So, for the Reading Development Continuum, the second digit of the code 
number means the following: 

1 = Role Play Phase 

2 = Experimental Phase 

3 = Early Reading Phase 

4 = Transitional Phase 

5 = Independent Phase 

Note that no data were collected for the Advanced Phase indicators because it was 
judged that no primary school child would exhibit the behaviours described by 
the indicators in this phase. 
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An example to help use the code numbers. The code number '2102' means 'the 
Reading Development Continuum', 'Role Play Phase', indicator '02', which is, 
'The child realises that print carries a message but may read the writing 
differently each time ..." 
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1. Indicators ordered by indicator code number. 



Reading 

Indicator 

Code 


Difficulty 

Estimate 


Role Play Reading Phase 


2102 


-3.21 


2103 


0.29 


2104 


-0.93 


2105 


0.11 


2106 


-4.17 


2107 


-2.87 


2108 


-2.08 


2109 


-2.87 


2111 


-2.08 


2112 


-1.66 


2115 


-4.17 


2116 


-5.00 


2117 


-1.28 


2118 


-0.93 


2119 


-1.66 


Experimental Reading Ph 


2201 


-1.91 


2202 


0.87 


2203 


-0.74 


2204 


-4.55 


2205 


-4.07 


2206 


0.42 


2207 


-2.60 
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Experimental Reading Phase (continued) 



Reading 

Indicator Difficulty 
Code Estimate 



2208 


-0.59 


2209 


-0.25 


2210 


0.36 


2211 


-1.52 


2212 


-0.26 


2213 


0.87 


2214 


-4.55 


2215 


-2.16 


2216 


-2.04 


2218 


-1.34 


2219 


-0.06 


2220 


-0.38 


2221 


-1.49 


2222 


-3.41 


2223 


-0.78 


2224 


-1.05 


2225 


-3.18 


2226 


-1.69 


Early Reading Phase 


2301 


0.53 


2302 


2.91 


2303 


-0.55 


2304 


-0.70 


2305 


0.32 


2306 


-0.99 


2307 


0.00 


2308 


1.53 


2309 


-0.94 


2310 


- 0.77 


2311 


0.76 


2312 


-2.35 
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Early Reading Phase (continued) 



Reading 

Indicator Difficulty 
Code Estimate 



2313 


-0.08 


2314 


-0.39 


2315 


-0.42 


2316 


-0.21 


2317 


0.00 


2318 


1.18 


2319 


1.65 


2320 


1.27 


2321 


-0.09 


2322 


-1.46 


2323 


-1.20 


2324 


-0.21 


2325 


-1.13 


2326 


-0.93 


2327 


-1.07 


Transitional 


Reading Phase 


2401 


1.83 


2402 


3.52 


2403 


1.11 


2404 


1.91 


2405 


0.72 


2406 


2.81 


2407 


2.66 


2408 


1.34 


2409 


1.27 


2410 


0.38 


2411 


0.60 


2412 


0.59 


2413 


1.45 


2414 


-0.06 


2415 


-0.56 
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Transitional Reading Phase (continued) 
Reading 

Indicator Difficulty 
Code Estimate 



2416 


0.50 


2417 


0.41 


2418 


1.10 


2419 


-Z10 


2420 


-0.19 


2421 


0.54 


2422 


1.21 


2423 


-0.82 


Independent 


Reading Phase 


2501 


3.69 


2502 


2.29 


2503 


4.39 


2504 


3.58 


2505 


0.99 


2506 


2.63 


2507 


3.25 


2508 


3.75 


2509 


4.40 


2510 


3.53 


2511 


2.71 


2512 


4.19 


2513 


3.69 


2514 


3.38 


2515 


0.04 


2516 


1.52 


2517 


-0.61 


2518 


1.77 


2519 


2.54 
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2. Indicators ordered by estimate of difficulty 
Reading 

Indicator Difficulty 
Code Estimate 



2116 


-5.00 


2204 


-4.55 


2214 


-4.55 


2106 


-4.17 


2115 


-4.17 


2205 


-4.07 


2222 


-3.41 


2102 


-3.21 


2225 


-3.18 


2107 


- 2.87 


2109 


-2.87 


2207 


-Z60 


2312 


-2.35 


2215 


-2.16 


2419 


-2.10 


2108 


-2.08 


2111 


-2.08 


2216 


-2.04 


2201 


-1.91 


2226 


-1.69 


2112 


-1.66 


2119 


-1.66 


2211 


-1.52 


2221 


-1.49 


2322 


-1.46 


2218 


-1.34 


2117 


-1.28 


2323 


-1.20 


2325 


-1.13 


2327 


-1.07 


2224 


-1.05 


2306 


-0.99 


2309 


-0.94 
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Reading Development Continuum Indicators ordered by estimate of difficulty 
(continued). 



Reading 

Indicator 

Code 


Difficulty 

Estimate 


2104 


-0.93 


2118 


-0.93 


2326 


-0.93 


2423 


-0.82 


2223 


-0,78 


2310 


-0.77 


2203 


-0.74 


2304 


-0.70 


2517 


-0.61 


2208 


-0.59 


2415 


-0.56 


2303 


-0.55 


2315 


-0.42 


2314 


-0.39 


2220 


-0.38 


2212 


-0.26 


2209 


-0.25 


2316 


-0.21 


2324 


-0.21 


2420 


-0.19 


2321 


-0.09 


2313 


-0.08 


2219 


-0.06 


2414 


-0.06 


2307 


0 


2317 


0 


2515 


0.04 


2105 


0.11 


2103 


0.29 


2305 


0.32 


2210 


0.36 


2410 


0.38 
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Reading Development Continuum Indicators ordered by estimate of difficulty 
(continued). 

Reading 

Indicator Difficulty 
Code Estimate 



2417 


0.41 


2206 


0.42 


2416 


0.50 


2301 


0.53 


2421 


0.54 


2412 


0.59 


2411 


0.60 


2405 


0.72 


2311 


0.76 


2202 


0.87 


2213 


0.87 


2505 


0.99 


2418 


1.10 


2403 


1.11 


2318 


1.18 


2422 


1.21 


2320 


1.27 


2409 


1.27 


2408 


1.34 


2413 


1.45 


2516 


1.52 


2308 


1.53 


2319 


1.65 


2518 


1.77 


2401 


1.83 


2404 


1.91 


2502 


2.29 


2519 


2.54 


2506 


2.63 


2407 


2.66 


2511 


2.71 


2406 


2.81 
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Reading Development Continuum Indicators ordered by estimate of difficulty 
(continued). 

Reading 

Indicator Difficulty 
Code Estimate 



2302 


2.91 


2507 


3.25 


2514 


3.38 


2402 


3.52 


2510 


3.53 


2504 


3.58 


2501 


3.69 


2513 


3.69 


2508 


3.75 


2512 


4.19 


2503 


4.39 


2509 


4.40 
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APPENDIX C: 



List of indicators form the Reading Development Continuum not understood by 
some Kindergarten, Year, 1, 3, 5 or 7 classroom teachers. 



This appendix lists the indicators which were not understood by one or more 
teachers. The indicators are identified by code numbers. See Appendix A to link 
the code numbers to the indicators. 

Number of 

Indicator Teachers not 

Code Understanding 



2218 1 

2301 1 

2302 1 

2311 1 

2318 1 

2401 2 

2402 2 

2404 1 

2406 1 

2412 1 

2413 1 

2414 1 

2417 1 

2418 2 

2420 1 

2421 1 

2423 1 

2502 1 

2503 2 

2507 3 
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APPENDIX D: 



Classroom teachers’ questionnaire about the First Steps continua. 

This appendix consists of the questionnaire which was used to ask classroom teachers 
about the First Steps Developmental Continua. 

The teachers could respond with information about any of the First Steps Continua, 
and not just to the Reading Developmental Continuum. 
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Australian Council for Educational Research, 

First Steps Evaluation, 1993 
Questionnaire for Classroom Teachers using First Steps 

The WA Ministry of Education has commissioned the Australian Council for Educational Research to 
evaluate the First Steps project. This questionnaire forms one part of the evaluation. 

You are one of a small number of classroom teachers chosen from ail WA government primary and 
district high schools. Because there are relatively few teachers being sampled, your response to 
this survey is especially important for understanding how teachers judge the adequacy of the First 
Steps Continua. 

Your participation in this survey is voluntary. You may refuse to answer any or all of the questions 
contained in this questionnaire. If you choose not to participate could you please contact me at the 
address below so that I do not send out reminder notices to you. 

Your responses to these questions will be treated confidentially and no schools 
or Individual respondents will be identified in any reports produced as part of this 
evaluation unless permission is given. (For more information see the access conditions to 
the data which are available from your school principal.) 

If you have any questions about this survey or the evaluation feel free to contact me by phone on 
(03) 819 1400, fax (03) 819 5502 or by mail at the address below. 

The questionnaire will take about 30 minutes to complete. Just how long it takes will depend on 
your involvement with the First Steps Project. When you have completed this questionnaire, please 
post it using the attached Reply Paid envelope. If you have misplaced the envelope, send the 
completed questionnaire to: 




Adrian Harvey-Beavis 
First Steps Evaluation 
Reply Post 2 

Australian Council for Education Research 
PO Box 210 
Hawthorn 
Victoria 3122 

(If you use this address you will not need a postage stamp.) 



Please post the completed questionnaire before 
Friday November 12 



Complete this survey form by ticking the appropriate box or by writing in the space 
provided. H Q 



1 . What Is the name of this school? 

(The name of your school is needed so that other information from other souroes can be 
linked to it, for example, whether it is an urban or rural or remote school It is also needed to 
check off returns so that you are not sent a reminder notice.) 



2 . What Year level Is the class that you are now teaching? 

(If it is a composite dass then write in all Year levels eg Years 4/5 ) 



About the First Steps continua 



3. During any time In 1993 have you used, with the students In the class you 
are currently teaching, any of the following First Steps Continua? 

(Tick as many boxes as apply) 



Writing 


Development 


Continuum 


Yes 

□ 


Writing Learning 


Continuum 


□ 


Spelling 


Development 


Continuum 


□ 


Reading 


Development 


Continuum 


□ 


Language 


Development 


Continuum 


□ 



If you have nol used any First Steps Continua with the class you are currently teaching ,at any time in 
1 993 . then stop here . Thank you for your help. 



4 . Comment on how well the Continua (which you are using with the class you 
are now teaching) depict the development that you see in your students? 
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5 . 



How might the First Steps Continua which you have used with the students, 
in the class you are currently teaching, be further Improved and 
developed? 



6 . Generally, how helpful do you find the First Steps Continua that you have 
used this year? 

Only tick boxes for those continua that you have been using in 1993 with the class that you 
are currently teaching. 





Very 

Helpful 


Helpful 


Neither 

helpful 

nor 

unhelpful 


Unhelpful 


Very 

unhelpful 


Writing Developmental 
Continuum 


□ 


□ 


□ 


□ 


□ 


Writing Learning 
Continuum 


□ 


□ 


□ 


□ 


□ 


Spelling Developmental 
Continuum 


□ 


□ 


□ 


□ 


□ 


Reading Developmental 
Continuum 


□ 


□ 


□ 


□ 


□ 


Oral Language 
Developmental 
Continuum 


□ 


□ 


□ 


□ 


□ 



There is a blank back page for you to write any further comments you may have 
about the First Steps Continua. 




Thank you for completing this survey. 
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